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WO 99/35852 



PCT/IB98/02094 



Group based control scheme for video compression. 



1 . Field of the Invention 

The present invention is directed toward a group based control scheme for 
video compression. 

5 2 . D escrip t ion o f Related Art 

Most of the body of work in this field utilizes networking based feedback to 
control video compression parameters. See, for example, Wakeman et al., AMulticast 
Congestion Control in the Distribution of Variable Rate Video on the Internet^ Proceedings 
of SigComm 1994 (hereinafter "Wakeman"). 

1 0 Other research has addressed AUser-Centered Video@ wherein attempts are 

made to control video compression parameters of different video streams based on the level of 
interest of each receiver in a system. Such a scheme works fine for a unicast based Avideo- 
on-demand@ system. It does not work, however, for multicast group based activities such as 
internet video conferencing applications. See, Yamaahi et al., AUser-Centered Video: 

15 Transmitting Video Images Based on the User=s Interest®, Proceedings of CHI 1995. 

U.S. Pat. No. 4,494,232 entitled AStatistical Multiplexer with dynamic 
Bandwidth Allocation for Asynchronous and Synchronous Channels® (1985) addresses 
multiplexing different streams of data, both synchronous and asynchronous, given certain 
control signals coming from data terminal equipment (DTE); however, bandwidth allocation 

20 to specific users/applications within an ongoing group session is not controlled. 

U.S. Pat. No, 5,600,797 entitled ASystem for Identifying New Client and 
Allocating Bandwidth Thereto by Monitoring Transmission of Message Received Periodically 
from Client Computers Informing of their Current Status® (1997) addresses the issue of 
initiating a communication session among different machines on a network through a central 

25 manager. The central manager also allocates the required bandwidth according to the 
reception of valid client requests. 

E.P, application 258,514 entitled AOptimized Bandwidth Allocation 
Mechanism Between Circuit Slots and Packet Bit Stream in a Communication Network® 
(1988), E.P. application 713,347 A2 entitled AMethod and Apparatus Enabling STM and 
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Packet Mode Access on a Broadband Communications Network@ (1996), and EJ\ 
application 479,4332 A2 entitled AMethod and Apparatus for Dynamic Channel Bandwidth 
Allocation Among Multiple Parallel Video Coders@(1992), all relate to very low levels e.g., 
ICs, packets, and parallel video coders, dynamic bandwidth allocation mechanisms on a low 
>hardware= level. 

The present invention comprises a method by which video compression 
parameters of different video streams are changed to optimize overall bandwidth utilization 
according to rules related to the varying interactions among group members. Different video 
streams are allocated different portions of the overall bandwidth based on a priority scheme. 
The priority scheme is based upon group dynamics. A controller (central or distributed) 
receives at least two types of signals: a network congestion control signal which indicates the 
overall quality of delivered video streams and whether more or less bandwidth needs to be 
allocated for that particular video stream, and a group dynamics monitoring signal which 
monitors the various activities of group members. The controller (central or distributed) uses 
the group dynamics monitoring signal to assign priorities assigned to the different video 
streams according to a set of rules supplied by the application. The controller then uses these 
priorities to divide the bandwidth available to the application, that is being supplied to the 
controller either directly through the network congestion control signal or calculated using it, 
among the different video streams. The new bandwidth portions are then sent to the different 
video encoders which should change their video output rate, by changing some controlling 
parameters within the Codec, to adhere to the ones sent by the controller. 

FIG. 1 illustrates a schematic of the network architecture of the various group 
members and controller (central or distributed). 

FIG. 2 is a more detailed illustration of the controller (central or distributed). 
FIG. 3 is a more detailed illustration of one of the group members. 
FIG. 4 is a block diagram illustrating the steps of the present invention. 
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The present invention shall now be described with reference to the 
accompanying figures wherein like numerals shall represent like elements in the various 
figures. 

Most groupware applications (e.g. video conferencing, distance learning) 
require transmission of multiple video streams among groups of people. Multimedia streams 
are generally compressed according to standards such as MPEG and H.261 for video, and 
PCM and GSM for audio. Despite compression, currently deployed networks usually do not 
have enough bandwidth capacity to allow continuous transmission of multiple high quality 
video streams. This is especially true for anyone accessing networks via modems or other 
dial-up type connections. 

Video compression algorithms such as MPEG and H.261 provide different 
levels of compression by using different quantization tables or changing the video frame rate. 
Different compression levels result in compressed streams having different data rates. 
Usually, multi-use video communication tools use these characteristics of compression 
algorithms to adapt their output data rate according to networking feedback. 

A method is desired by which video compression parameters of different video 
streams are changed to optimize overall bandwidth utilization according to rules related to the 
varying interactions among group members. In other words, given a limited bandwidth, 
different video streams are allocated different portions of the bandwidth based on a priority 
scheme. 

According to the present invention, there is a minimum data rate allocated for 
each video stream in the system. Below this minimum data rate the video stream does not 
convey meaningful information to members in the group. The minimum data rates are content 
based and are best set by the video application (group member) that is sending the video 
stream. Member interactions and/or group communication sessions are dynamically analyzed 
and result in generation of high level signals that can be used as a feedback controller for 
video data rate thereby allowing video communication to take place on low bandwidth links 
while improving video quality on high bandwidth links. 

An example would be when only one group member is talking for a long time 
while other group members are not generating any extraordinary activity. The video encoder 
of the talkative group member would start generating high bandwidth video while the other 
group members encoder=s can be temporarily degraded. 

A controller (central or distributed) receives at least two types of inputs, The 
controller, if distributed, can be an integral part of the application or a separate part that ends 
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feedback to the application through some interprocess communication mechanism. One is a 
network congestion control signal which may indicate overall quality of delivered video 
streams and whether more or less bandwidth needs to be allocated for that particular video 
stream. A second signal is a group dynamics monitoring signal which monitors various 
activities of group members. Activities include audio, video, data, keyboard, mouse, etc. This 
high level signal describing the status (e.g., dominating, active, inactive) of each group 
member is generated and then received on each group member=s activity. 

The network congestion control signal is typically used to estimate the overall 
bandwidth available for an application and all of its video streams, such as a signal described 
in "Wakeman". This signal can differ based upon the type of network. In a network that 
supports resource reservation so that bandwidth can be reserved by the application (e.g. 
ATM), this signal will be static. In a network that does not support resource reservation (e.g. 
internet), this signal can be a congestion control signal reporting overall congestion status of 
the group. This signal can then be used to determine the currently available bandwidth that 
can be used by the application. In this case the signal is dynamic. 

The controller then assigns priorities to the different video streams of the group 
members based upon the status determined by the group dynamics monitoring signal. The 
controller uses priorities to divide bandwidth available to the application, supplied directly or 
indirectly through the network congestion signal, between different video streams. The 
assigning process is preferably based upon the nature of the group activity or supplied by the 
application. For instance, some activities such as teaching may assign an inactive group 
member a high priority while other activities such as a business meeting would assign an 
inactive group member a low priority. 

In an example implementation of "application rules", two tables are used. A 
first table, a table of conditions, specifies conditions that trigger a change of priorities. A 
second table, a table of actions, specifies for each condition, what priority change should be 
performed. The concept of >roles= may also be used in the process of assigning video stream 
priorities. 

Consider a teaching session where a teacher and a group of students come 
together to discuss an assignment. Only the audio signal is going to be used to describe the 
group activities. There are two roles, teacher and student. A session management-Hke 
protocol is desired so that the controller can know about new users in terms of identifying each 
user, and how to send/receive data from that user=s role. If the controller is to be implemented 
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as part of an application then the session management protocol used by the application can 
also be used by the controller. Otherwise, a specific protocol needs to be implemented. 
. An example table of conditions below is based on "roles". 



TABLE 1 



User — Role 


Signal 


Condition 


ALL 


Audio OFF (INACTIVE) 


1 


Teacher 


Audio ON (ACTIVE) 


2 


Student 


Audio ON (ACTIVE) 


3 


ANY 


Audio OFF (INACTIVE) 


4 



An example table of actions is shown below. 



TABLE 2 



Condition 


User - Signal 


Priority 


1 


ALL INACTIVE 


Initialize 


2 


ACTIVE Teacher 


Increase (5) 


3 


ACTIVE Student 


Increase (2) 


4 


ANY INACTIVE 


Decrease (I) 



In this example, priorities are within a preset range, for example 1-10. When an 
10 OFF Audio signal is received from ALL users* all user priorities will initialized, in this 
example, to equal priorities. If an Audio ON signal (ACTIVE) is received from the user 
having the Teacher role t the priority of that user is increased by a preset amount, in this 
example, 5. If an Audio ON signal (ACTIVE) is received from a user having a Student role, 
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the user=s priority is increased by a preset amount, in this example, 2. Lastly, if an Audio 
OFF (INACTIVE) signal is received from ANY of the users, then the priority of that user is 
decreased by 1 . 

The controller will analyze group dynamics monitoring signals received from 
each user within a specific time window and based on this information, specify which 
conditions within the table of conditions are triggered. Action will be taken in response to the 
information acquired from the table of conditions. The bandwidth will then be divided among 
different video streams according to the relevant priorities. 

Roles need not be utilized, but instead, each activity by a user could change the 
priority. Similarly, the number of conditions, etc. and priority levels could easily be altered. 

The controller uses the priorities assigned to the different video streams (and, in 
this example, the network congestion control signal) to send feedback signals back to the 
different video transmitters via video encoder requesting the transmitters to either adhere to, 
increase, or decrease their respective video data rates by changing parameters controlling the 
video encoding process. Parameters may include the video resolution and/or frame rate 
among others well known in the art. If the minimum data rate for a particular video stream 
cannot be achieved without sacrificing higher priority video streams, then the controller will 
send a stop signal to the relevant video transmitter. In this example having a stop signal, an 
assumption is made that each user corresponds to one video stream although this may easily be 
altered. Each time a new group dynamics monitoring signal is received the controller begins 
the priority assigning process and feedback generation again. 

FIG. 1 illustrates the preferred embodiment of the invention. A controller 
(central or distributed) 100 is accessed and accesses a plurality of group members 200. Group 
members include applications and/or devices capable of sending and receiving video streams 
(or other types of data streams, including audio, video, and/or data, etc.). Thus, a group 
member may comprise a user operating a personal computer, a video teleconferencing device 
or any other device or application that can send and receive video streams. The various group 
members 200 are linked to one another such that each individual group member can broadcast 
or send to all others and may receive video streams from any of the other group members. 
Thus, this example is a multi-cast group based environment. 

In addition to sending and receiving video streams, group members 200 can 
also send and receive control signals to controller 100. Each group member 200 periodically 
sends a group dynamics monitoring signal to controller 100 informing controller 100 of video 
stream activity currently taking place. Controller 100 processes each group dynamics 
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monitoring signal in order to determine the optimum bandwidth requirement of each group 
member 200 and sends a feedback control signal instructing each group member as to how to 
re-configure itself for optimum bandwidth utilization. The network congestion control signal 
may also be used in this determination. 

FIG. 2 is a more detailed diagram of controller 100. The group dynamics 
monitoring signals from the various group members are received into a module 110 that 
assigns a status to each group member based upon the activity of that group member. The 
status data is forwarded to a module 120 which also, in this example, receives the network 
congestion control signal. Module 120 then determines the bandwidth requirements and 
assigns priorities for each group member based on the received status data. Network 
congestion data may also be considered. Finally, the priorities and updated bandwidth 
allocations are sent back to each group member. 

FIG. 3 is a more detailed diagram of a representative group member 200. Each 
group member 200 receives a pair of inputs, inbound video stream data from other group 
members and a feedback signal from the controller 100. Moreover, each group member 200 
sends at least two outputs, a group dynamics monitoring signal to the controller 100 and 
outbound video stream data for the other group members. The incoming feedback signal is 
routed to video encoder 210 for adjusting the data rates of video before being forwarded to 
transmitter 220 for transmission to the group. Module 230 sets the minimum data rate needed 
to convey meaningful data to other users. 

FIG, 4 is a block diagram illustrating the processes involved. Initially, the 
group members set minimum data rates 410 for their respective video streams. In this 
example, the network congestion control signal is checked 412 and the various group 
dynamics monitoring signals are evaluated 414. Upon evaluation, the controller assigns a 
status value 416 to each group member and determines the bandwidth requirements of each 
group member 418. Next, the controller assigns priorities 420 and allocates bandwidth 422 to 
the group members. A check 422 by the controller is performed to ensure that each group 
member is at or above their minimum data rate. If so, then the controller sends 428 the 
feedback signals to the group members. If not, then the controller will issue a stop signal 426 
to the relevant group member before sending the feedback signals 428 to the group members. 
The group members then process the feedback signals 430 and change their video encoding 
parameters accordingly prior to sending out 432 their respective video streams. The process is 
started over again upon receipt of a new network congestion control signal and/or a new set of 
group dynamics monitoring signals. 
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Although the embodiment described utilizes the network congestion control 
signal, clearly this signal could be removed and not considered. 

It is to be understood that the foregoing disclosure taught and described herein 
is illustrative of the present invention. Modifications may readily be devised by those 
ordinarily skilled in the art without departing from the spirit or scope of the present invention. 
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1 , A system for bandwidth allocation to a group of devices (200) or applications 
comprising; 

a plurality of group member devices or applications capable of video stream 
transmission and reception, each group member capable of broadcasting video streams to the 
5 entire group and receiving video streams from a group, said group member devices or 
applications linked together in a network; and 

a controller (100) for receiving group dynamic monitoring signals from each 
group member device or application, said group dynamic monitoring signals indicating the 
activity of a group member and for determining bandwidth requirements of a group member 
10 device or application and allocating a bandwidth, via a feedback signal to said group member 
device or application, based on the activity of said group member devices or applications. 

2. The system of claim 1, wherein said controller is also linked to said network 
and receives a network congestion control signal from said network indicating the overall 

15 quality of delivered video streams, said network congestion control signal for use, along with 
said group dynamic monitoring signals, in determining bandwidth requirements of each group 
member device or application and allocating (130) the bandwidth for each group member 
device or application. 

20 3, The system of claim 1 , wherein each group member device or application sets a 

content based minimum data rate bandwidth requirement (230) which it requires to send a 
video stream and forwards its minimum data rate bandwidth requirement to the controller. 



4. The system of claim 3, wherein the controller checks whether each group 

25 member=s minimum data rate bandwidth requirement is met (110, 120) and, if a group 

member=s minimum data rate bandwidth requirement cannot be achieved without sacrificing a 
higher priority group member=s data rate bandwidth requirements, said controller sends a stop 
signal to the relevant group member device or application. 



10 
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5. The system of claim 4, wherein said network congestion control signal is 
periodically updated in the controller and each time a new network congestion control signal is 
updated, the controller re-determines and re-allocates (120, 130) the bandwidth requirements 
of each group member device or application. 

6. The system of claim 4, wherein said group dynamics monitoring signals are 
periodically updated by each group member device or application and each time a new group 
dynamics monitoring signal is sent to the controller, the controller re-determines and re- 
allocates the bandwidth requirements of each group member device or application. 

7. The system of claim 6, wherein said network is any video or audio 
communications network. 



8. The system of claim 7, wherein said network is an internet based video or audio 
1 5 communications network, 

9. A method of bandwidth allocation to a group of devices (200) or applications 
comprising the steps of: 

a. monitoring (110, 120) the activity of a plurality of group members engaged in a 
20 multi-cast video stream communication network; 

b. monitoring the congestion of the multi-cast video stream communication 
network; 

c. determining and assigning (120) a status and priority to each group member 
based on the activities of the group as a whole; 

25 d, allocating a bandwidth (1 30) to each group member based upon said status and 

priority; and 

e. periodically updating (120) the status and priority values of each group 

member based on continued monitoring of network congestion and group activity, 

30 10. A controller for receiving group dynamics monitoring signals from a plurality 

of group member devices or applications being capable of transmitting video streams to a 
group, said group dynamic monitoring signals indicating the activity of said group members, 
and for determining bandwidth requirements of each group member device or application and 
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allocating a bandwidth via a feedback signal to each group member device or application 
based on the activity of the group member devices or applications. 
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